Contrastive learning with token projection for Omicron pneumonia identification from few-shot chest CT images

Introduction Deep learning-based methods can promote and save critical time for the diagnosis of pneumonia from computed tomography (CT) images of the chest, where the methods usually rely on large amounts of labeled data to learn good visual representations. However, medical images are difficult to obtain and need to be labeled by professional radiologists. Methods To address this issue, a novel contrastive learning model with token projection, namely CoTP, is proposed for improving the diagnostic quality of few-shot chest CT images. Specifically, (1) we utilize solely unlabeled data for fitting CoTP, along with a small number of labeled samples for fine-tuning, (2) we present a new Omicron dataset and modify the data augmentation strategy, i.e., random Poisson noise perturbation for the CT interpretation task, and (3) token projection is utilized to further improve the quality of the global visual representations. Results The ResNet50 pre-trained by CoTP attained accuracy (ACC) of 92.35%, sensitivity (SEN) of 92.96%, precision (PRE) of 91.54%, and the area under the receiver-operating characteristics curve (AUC) of 98.90% on the presented Omicron dataset. On the contrary, the ResNet50 without pre-training achieved ACC, SEN, PRE, and AUC of 77.61, 77.90, 76.69, and 85.66%, respectively. Conclusion Extensive experiments reveal that a model pre-trained by CoTP greatly outperforms that without pre-training. The CoTP can improve the efficacy of diagnosis and reduce the heavy workload of radiologists for screening of Omicron pneumonia.


Introduction
In the tail of February 2022, a new round of COVID-19 epidemic caused by subvariant Omicron BA. 2 and BA.2.2 broke out in Shanghai (1).There are more than 30 mutation sites in the spike protein of the Omicron mutant, which increases the binding ability of the virus to human cells, and the infectivity is 37.5% higher than that of the Delta variant (2,3).Until 1 June 2022, Omicron had caused 626,811 infection cases, including 568,811 asymptomatic infections, 58,000 symptomatic cases, and 588 deaths (4), which brought great crisis and challenge to social public health security (5).
Currently, the real-time reverse-transcriptase-polymerase-chainreaction (RT-PCR) test is the main diagnostic tool (6), while chest CT imaging is increasingly recognized as a complementary or even a reliable alternative method (7,8).Figure 1 illustrates some CT scan images of mild and severe Omicron pneumonia.All annotations have been provided by experienced doctors, who evaluate patients based on their clinical conditions and CT imaging.From that, we can find that the CT images of mild Omicron pneumonia usually show slight inflammation in the lungs.On the contrary, the CT images of severe Omicron pneumonia are more serious than those of mild Omicron pneumonia, showing more severe inflammation and damage to the lungs.Physicians need to pay more attention to patients with severe Omicron pneumonia and treat them in time.However, experienced radiologists are needed to manually identify all the thin-slice CT images (an average of 300 layers per patient) (9).This may lead to misdiagnosis due to the significantly increased workload of radiologists.
With the development of deep learning (10), researchers can extract useful information from a significant volume of annotated data (11).However, when compared to natural images, acquiring such quantities of medical data is challenging, and the annotations must be carried out by professional radiologists (12,13).This poses huge challenges to applying deep learning to medical image analysis and processing (14).In recent years, contrastive learning methods (15)(16)(17)(18)(19) have achieved satisfactory results in natural image classification tasks.These methods can utilize unlabeled data to create a pre-trained model, which can then be fine-tuned with lightly annotated data for further improvement.
While some studies have investigated the effects of contrastive learning on natural image classification tasks, there remains a gap in research that specifically addresses chest CT images.The current methods based on contrast learning are insufficient in enhancing chest CT images effectively and exploring global features.To address this issue, we propose a novel contrastive learning with token projection, namely CoTP, to improve global visual representation.The token projection typically consists of a multi-head self-attention (MHSA) (20) and a fully connected (FC) layer.The MHSA can capture short and long-range visual dependencies, while the FC layer can eliminate redundant features.Moreover, we leverage the downsampling layer to reduce the cost of computation.In addition, a private Omicron dataset collected by the Geriatric Medical Center, Zhongshan Hospital, Fudan University is utilized for CoTP pre-training.Especially, data augmentations have important roles in contrastive learning methods (15).However, the widely used augmentations in contrastive learning approaches for natural images may not be suitable for chest CT.Therefore, a new data augmentation approach, random Poisson noise perturbation (PNP) is proposed for CT images, to simulate the noise in CT images.After pre-training, the feature encoder with pre-trained weights is taken out, followed by a simple max pooling and average pooling (MAP) head which can obtain different space areas occupied by objects of different categories.Then, we fine-tune the model on a sub-dataset extracted from Omicron datasets and the external SARS-CoV-2 CT-scan dataset (21), respectively.Extensive experiments reveal that a model pre-trained by the proposed CoTP greatly outperforms that without pre-training.Our main contributions to this work are summarized as follows: (1) A novel contrastive learning with token projection, namely CoTP, is proposed to improve the diagnostic quality of few-shot Omicron chest CT images.In particular, token projection with a downsampling layer is utilized to further improve the quality of the global visual representations and reduce the computational cost.(23) diagnosed COVID-19 via the proposed network with a multi-receptive field attention module on CT images.Moreover, Mei et al. (22) adopted ResNet (23) to rapidly diagnose COVID-19 patients using both full CT scans and non-image information.In addition, several works (24-26) also used segmentation techniques for detection.However, the current deep learning-based approaches for pneumonia diagnosis primarily rely on supervised learning, leveraging abundant labeled data to acquire precise visual representations.On the contrary, there are few-shot labeled chest CT images.Table 1 summarizes the previous studies on supervised learning for pneumonia diagnosis from chest CT images.

Contrastive learning in image analysis
Given the efficient visual representation ability of deep learning, contrastive learning has emerged as a promising approach for efficiently extracting accurate visual representations from unlabeled images (29).Wu et al. (16) first designed a framework that pulls away augmented views of different images (negative pair) while pulling in different augmented views of the same image (positive pair).Based on this idea, the two methods, SimCLRv1 (15) and MoCo-v1 (18) were proposed, which can greatly narrow the gap between supervised learning and unsupervised learning on downstream task performance.These methods, SimCLRv2 (17) and MoCo-v2 (19) employed the projection head to improve the ability of visual representation extraction and outperformed the supervised learning on downstream tasks.The success of these methods motivated many researchers to introduce contrastive learning into medical image analysis.Sowrirajan et al. (13) utilized MoCo pre-training to improve the representation and transferability of chest X-ray Models.Zhang et al. (30) obtained medical visual representations according to contrastive learning with paired images and texts.In addition, the works of various researchers (30)(31)(32) employed contrastive learning for medical image segmentation.However, the existing contrastive mechanisms have scope for improvement for Omicron pneumonia diagnosis from chest CT images due to their inability to mine global features and lack of appropriate augmentations for chest CT images.We present the pros and cons of previous studies on contrastive learning in image analysis in Table 2.
3 Materials and methods

The pipeline for interpretation of CT images
The overall pipeline for CoTP pre-training and the subsequent fine-tuning with CT images are illustrated in Figure 2.There are two stages for CT image interpretation: the CoTP pre-training stage and subsequent fine-tuning.First, we converted and exported the DICOM files of Omicron patients into JPEG formats and employed CoTP to pre-train the feature encoder using unlabeled Omicron CT images.Second, the feature encoder with pre-trained weights was taken out, followed by a simple linear classifier.Then, we fine-tuned the baseline with a few labeled CT images.

Random Poisson noise perturbation
Data augmentation is widely used in contrastive learning and is crucial for learning good representations (15).Nevertheless, most existing natural image data augmentations may not be suitable for chest CT images.For example, random crops and cutouts may remove or mask the lesion area of CT images.Meanwhile, color jitter and random grayscale transformation are no longer applicable to grayscale CT images.
As shown in Figure 3, we not only utilized traditional methods, i.e., random horizontal flipping, random center crop, and random rotation (10 degrees) but also a new data augmentation approach, random Poisson noise perturbation for CT images.Poisson distributed noise is a well-known data augmentation (34, 35).However, this was the first time that Poisson-distributed noise was applied to contrastive learning instead of Gaussian noise perturbation.In the process of scanning CT images, various noises will be generated due to the

Authors
Year published Pros.

Wu et al. (16) 2018
Maximize the distinction between instances and non-parametric instance discrimination.
Compared to supervised models, it still has a significant gap.( ) ( ) ( )

54.0% ACC on
Then, I n has to be processed with a logarithm and transform (iFanbeam) from the classical filtered back-projection (FBP) algorithm (38) to the image domain ′ X , as Eq. ( 2).Thus, we gained the Poisson noise CT image according to Eqs. (1, 2) as a more realistic data augmentation. (2)

Overview of the proposed CoTP
Algorithm 1 Pseudocode of CoTP.
Input: batch size B, constant temperature τ , negative memory bank , , ,• , encoder networks for query and key E q _ , E k _ , Token projection for query and key T q _ , T k _ , draw two augmentation functions CTAug1, CTAug2 # augmentation for query # augmentation for key

Feature encoder
As shown in Figure 4, we designed CoTP to learn global visual representations effectively from unlabeled CT images.Given a CT image, X , we utilized two different augmentations to create two views of the same example, V q and V k .Then, we employed ResNet50 (23) which removed the entire global pooling and Multilayer Perceptron (MLP) parts as the feature encoder.The V q and V k are mapped via encoders (q) and (k), to generate visual representations F q  , respectively.Here, H , W , and C are the length, width, and dimension of the feature map.The pseudocode of CoTP is shown in Algorithm 1.

Token projection
Traditional contrastive learning (17,19) typically uses a global pooling operation and an MLP as a projection head to improve the  To begin with, we reshaped the feature . Then, T was passed through three different linear projections and yielded a query Q, a key K , and a value V .Furthermore, we performed average pooling with a pooling size of S for K and V to reduce the cost of computation.Here, we set S = 7.After that, a convolution with 1 kernel size and 1 stride size was utilized to fuse the feature.Then, we gained the after performing layer normalization (LN) and ReLU activation functions.Afterward, we performed multi-head selfattention (MHSA) on Q, K, and V , as shown in Eq. ( 3), Then, we calculated the mean score of along the dimension of the column.Finally, we performed a linear projection to eliminate redundant features and obtain In particular, we set the dimension D = 128.

Update the weights
To meet a large number of negative sample pairs and reduce the computing cost of Graphics Processing Unit (GPU), a memory bank was used to store the negative samples generated by the encoder (q), in advance.Hence, we obtained a set of encoded (k) samples Out of all the encoded (k) samples in the set E k for each encoded query Z q , a single positive key Z k was matched, while the remainder of the keys (negative keys) represented different images.A contrastive loss function is represented in Eq. ( 4) as follows, whose value is low when Z q is close to its positive key Z k and moves away from all other encoded (k) samples: where τ is a temperature hyper-para (16), and the number of negatives N is set at 32,256.We updated the weights ω q of the encoder (q) and token projection (q) by back-propagation, while the weights ω k of the encoder (k) and token projection (k) were updated by momentum update (18), as Eq. ( 5), where m is 0.999 to update the weights slowly.

Subsequent fine-tuning
After CoTP pre-training, we took out the feature encoder with pre-trained weights, followed by a max pooling and average pooling (MAP) head, as shown in Figure 6.First, we performed a 1 1 × convolution and reshaped the feature Here, C denotes the class of categories.Afterward, we calculated the mean score and maximum score of the T along the dimension of the column, respectively.Finally, a Hyper-parameter » was employed to combine the mean score S a and maximum score S m , as Eq.(6).
It is noteworthy that max-pooling can be considered as classspecific attention that can attain different space areas occupied by objects of different categories.In particular, we performed a simple cross-entropy loss to fine-tune the baseline with a few-shot labeled CT images.

Results
The classification performances of the proposed methods were evaluated in terms of the standard metrics, such as accuracy (ACC), sensitivity (SEN), and precision (PRE) discussed in Eq. ( 7)-( 9), where P, N, TP, TN, and FP denote positives, negatives, true positives, true negatives, and false positives, respectively.

ACC = + +
TP TN N P (7) In addition, the mean AUC (39) was employed to evaluate the ability of the model to discriminate between different classes.Furthermore, we also used a non-parametric bootstrap (40) to estimate the variability around model performance.We performed a total of 500 bootstrap sampling with 300 cases from the test set.

Datasets
The study was approved by Zhongshan Hospital, Fudan University in Shanghai, China.All the chest CT scanning images in the Omicron The backbone of the subsequent fine-tuning.We leverage max pooling head and average pooling head to improve classification performance.The pipeline of token projection.The downsampling layer is designed for the reduction of calculation by reducing the resolution of feature.Following the above standards, we retrospectively collected highresolution CT images of 73 patients with mild Omicron pneumonia and 56 patients with severe Omicron pneumonia.The Omicron dataset and demographic characteristics of patients are detailed in Table 4. Initially, we converted and exported the DICOM files of Omicron patients into JPEG formats with 1,500 HU window width and 750 HU window level.After that, we obtained 50,500 unlabeled CT images with the size of 224 224 × for CoTP pre-training.Two experienced radiologists were selected, and they first labeled 2,742 CT images from the Omicron dataset.Then, we used the remaining data for CoTP pre-training.Note that the labeled CT images were excluded from the CoTP pre-training.The distribution of training and testing set in the Omicron dataset is shown in Table 5.In addition, we utilize the external SARS-CoV-2 CT-scan dataset presented by Soares et al. (21) to evaluate the transferability of CoTP.As shown in Figure 7, 1,252 CT scans were positive for SARS-CoV-2 infection (COVID-19), while 1,229 CT scanning for patients non-infected by SARS-CoV-2.

Transfer performance of CoTP representations for omicron pneumonia diagnosis
To assess the effectiveness of the visual representations extracted by CoTP, we employed VGG16 (41), DenseNet121 (42), and ResNet50 (23), as our backbones and selected six types of pre-training methods for comparison.As depicted in Table 6, the non-pre-training method's weights were randomly initialized, while the supervised pre-training method underwent pre-training on ImageNet-1k (43).In addition, we presented a more comprehensive comparison with the existing contrastive methods, such as SimCLRv1 ( 15), MoCo-v1 (18), SimCLRv2 (17), and MoCo-v2 (19) to prove the effectiveness of the proposed CoTP method.We evaluated the classification performance of the model using mean AUC, accuracy (ACC), sensitivity (SEN), and precision (PRE) of each infection type.Based on Table 6 and Figure 8 we gained the following observations:   As shown in Figure 9, we measured the dispersion of the test set using a box plot and performed statistical significance testing using a paired t-test.From this, we observed that CoTP significantly outperforms the non-pre-training method (p-value < le-5) and the supervised method (p-value < le-5), while performing slightly better than contrastive learning MoCo-v2 (19) (p-value < le-4).Moreover, it can be seen that our CoTP method had better robustness than other types of pre-training methods since the distribution of AUCs was more concentrated.

Transfer benefit of CoTP pre-training on an external SARS-CoV-2 CT-scan dataset
We conducted experiments to test whether CoTP pre-trained chest CT representations acquired from a source dataset (Omicron dataset) transfer to an external dataset, the SARS-CoV-2 CT-scan dataset.Table 8

Discussion
Recently, contrastive learning methods have achieved satisfactory results on natural image classification tasks, which can leverage unlabeled data to generate a pre-trained model.However, the existing contrastive mechanisms have scope for improvement for Omicron pneumonia diagnosis from chest CT images due to their inability to mine global features and lack of appropriate augmentations for chest CT images.Therefore, we proposed a novel contrastive learning with token projection, namely CoTP, for improving global visual representation.Furthermore, we leveraged a new data augmentation approach, random Poisson noise perturbation (PNP) to simulate the noise in CT images which is more realistic.In this section, we designed comprehensive ablation studies to assess the effectiveness of each component in the CoTP network.

Statistical significance testing for baseline characteristics of patients
First, we utilized the chi-square test for statistical significance test for baseline characteristics of patients, including age and

Effects of using different encoders on CoTP performance
Then, we leveraged several encoders VGG16 (41), ResNet50 (23), DenseNet121 (42), and Swin-B (51) for performance comparison, as shown in Figure 11.The results indicated that both convolutional neural networks (CNN)-based encoders and transformer-based encoders exhibited higher AUC and better robustness after pre-training with CoTP.Moreover, it can be seen that there are significant differences between pre-training methods (i.e., P ≤ le-5 between supervised pre-training method and CoTP when using ResNet50 as a backbone).

Impact of the random Poisson noise perturbation on CoTP performance
In addition, we investigated the impact of the proposed Poisson noise perturbation (PNP) during the data augmentation process.Therefore, we compared the model performance with and without the PNP and also compared it with the random Gaussian noise perturbation (GNP).From Table 9, we found that PNP affected the performance.For example, the VGG16 (41) with PNP could achieve an accuracy of 0.79%, a sensitivity of 0.84, and a precision of 0.88%, which are higher than those without PNP.On the contrary, GNP could not significantly improve the performance of the model.The PNP could simulate the noise CT images, which can improve the generalization of the model.

Impact of the MAP head on subsequent fine-tuning performance
To evaluate the ability of the MAP head on subsequent fine-tuning performance, the traditional classification (TC) head, which typically consists of a global pooling operation and a fully connected layer, was used for comparative experiments on the SARS-CoV-2 CT-scan dataset.Here, we used ResNet50 (23) pre-trained by our CoTP as the backbone.Based on Figure 12, we found that the proposed MAP head outperforms the TC head and achieved the best overall performance with » = 0.02.

Impact of the training data size
To study the transferable ability of the model under limited labels during the fine-tuning phase, we experimented with 10, 25, 50, 75, and 100% training data size on the SARS-CoV-2 CT-scan dataset.As shown in Figure 13, we illustrated three types of pre-training methods based on ResNet50 (23).The weight of none pre-training method was randomly initialized, and the supervised pre-training method was pre-trained on ImageNet.From the results, it can be inferred that the expected trend of improving ACC is with an increase in labeled data for the fine-tuning phase.Moreover, it is promising to observe that even with a 50% training data size, the CoTP asymptotically approaches the fully supervised (100% training data size) setup.

Visualization of grad-CAM heat map
Finally, we illustrated Grad-CAM (52) visualizations of the features learned by different pre-training methods based on ResNet50 in Figure 14.The higher response was highlighted in red while the lower one was demonstrated in blue.The expert annotation of the infected regions was indicated by a red dotted circle.As can be seen, the heatmaps generated by non-pretraining method are fuzzy and blurred.In addition, the heatmaps yielded by the supervised pre-training method focused on the edge areas of CT images.On the contrary, our CoTP learned more features that focus on the infection region, which can improve the classification accuracy in comparison with approaches.The highest scores in each model are shown in boldface.

Comparison of inference efficiency
Effect of the MAP head on subsequent fine-tuning performance.

Conclusion
The existing contrastive mechanisms have scope for improvement for Omicron pneumonia diagnosis from chest CT images due to their inability to mine global features and lack of appropriate augmentations for chest CT images.Therefore, we proposed a novel contrastive learning model with token projection, namely CoTP, for improving few-shot Omicron chest CT image diagnostic quality.Specifically, we designed the token projection to extract the global visual representation from unlabeled CT images.Furthermore, we leveraged random Poisson noise perturbation to simulate the noise CT images as a novel data augmentation.In addition, the MAP head which can obtain different  Extensive experiments on collected datasets demonstrated that our CoTP can provide high-quality representations and transferable initializations for CT image interpretation.In the future, we plan to design more effective pretext tasks and apply the proposed method to more medical image analysis tasks.For image segmentation and edge detection, we can employ the pre-trained encoder as a feature extraction, and then add a segmentation head or a detection head.

Funding
The author(s) declare financial support was received for the research, authorship, and/or publication of this article.The authors greatly appreciate the financial support from the National Natural Science Foundation of China (81971863, 82170110), the Shanghai Natural Science Foundation (22ZR1444700), the Shanghai Shenkang project for transformation for scientific production (SHDC2022CRD049), the Fujian Province Department of Science and Technology (2022D014), the Shanghai Pujiang Program (20PJ1402400), the Science and Technology Commission of Shanghai Municipality (20DZ2254400, 20DZ2261200), the Shanghai Municipal Science and Technology Major Project (ZD2021CY001), and the Shanghai Municipal Key Clinical Specialty (shslczdzk02201).

FIGURE 1 CT
FIGURE 1 CT scan images of mild and severe Omicron pneumonia.Severe Omicron pneumonia areas are marked with red dotted circles.(A) Mild Omicron pneumonia.(B) Severe Omicron pneumonia.

FIGURE 2
FIGURE 2Overview of Omicron pneumonia.

FIGURE 3
FIGURE 3Illustrations of a series of CT augmentation methods, i.e., random horizontal flipping, random center crop, random rotation (10 degrees), and the proposed random Poisson noise perturbation.

FIGURE 4
FIGURE 4The overall architecture of the proposed CoTP.
demonstrates the classification results of the previous methods (27, 28, 44-50) and six types of pre-training methods based on ResNet50, while the confusion matrices for four of them are shown in Figure 10.Based on these various experimental results, we can draw the following conclusions: (1) Visual representations learned from CoTP have better transferability in downstream tasks than those from ImageNet pre-training.(2) By taking advantage of the ability of our CoTP pre-training, our model outperforms all other contrastive methods on all metrics with a large margin in discriminating between COVID and non-COVID from CT images.For example, the ACC score of CoTP increased by 7.25 and 1.01% comparing the non-pretraining and MoCo-v2 pre-training, respectively.
To assess the inference efficiency, we calculated the pre-training time and parameters of MoCo-v1 (18), SimCLRv1 (15), MoCo-v2(19), and our CoTP on the Omicron dataset.As shown in Table 10, we obtained the following observations: (1) The parameters of the methods are nearly identical.MoCo-v2 (19) adds a simple linear projection based on ResNet50 (23).Meanwhile, our CoTP included an efficient token projection in addition to ResNet50 (23).(2) The training time of SimCLRv1 (15) is the shortest among the methods because it does not utilize a memory bank.(3) Although our CoTP slightly exceeds other methods in terms of training time and parameters, it achieves the highest accuracy of 92.35% and significantly outperforms the other methods.

FIGURE 11 Box
FIGURE 11Box plot of AUC values produced by different encoders and types of pre-training methods based on the Omicron dataset.We use 500 bootstrap samples with 300 cases to calculate the AUC which can evaluate the robustness of the model.

FIGURE 13 Effect
FIGURE 13Effect of training data size on model performance.
We verify the effectiveness of the proposed CoTP on the private Omicron dataset and the external SARS-CoV-2 CT-scan dataset, which delivers promising results on both datasets.Since the outbreak of coronavirus disease COVID-19 was declared a pandemic by the WHO on 11 March, 2020, various deep learning-based methods have been implemented worldwide to promote and save critical time for pneumonia diagnosis from CT images.Wu et al. (9) proposed a multi-view fusion model to improve the efficacy of diagnosis.A previous study Mei et al. (22) designed a grad-CAM-based deep learning method for fast detection of COVID-19 cases.Another study

TABLE 1
Previous studies on supervised learning for pneumonia diagnosis from Chest CT images.

TABLE 2
Previous studies on contrastive learning in image analysis.
and the noise distribution is more accurately characterized by the Poisson distribution (36).Consequently, we employed random Poisson noise perturbation to simulate the noise in CT images as a new data augmentation for CT images.First, we performed fan beam projection (37) transformation on the CT image X , and added Poisson noise as Eq.(1), where b stands for the number of photons.Here b is set as le-6.
(32) et al.(32)2021Generate contrastive data pairs based on the position of a slice in volumetric medical images.lack of appropriate augmentations for medical images.92.9% Dice on the ACDC dataset.Wu et al. (33) 2022 The proposed network does not rely on large negative samples.Lack of global visual representation.89.4% Dice on the ACDC dataset.10.3389/fmed.2024.1360143Frontiers in Medicine 05 frontiersin.orgphotoelectric interaction,

TABLE 3
The working environment.

TABLE 4
Demographics and baseline characteristics of patients in the Omicron dataset.

TABLE 5
Distribution of training and testing set in Omicron dataset.

TABLE 6
The transfer results of Omicron pneumonia diagnosis.It is noted that we need to pay more attention to elderly patients since they are more vulnerable to severe Omicron pneumonia.
gender.Based on Table7, we can see that there is no significant difference in gender (p = 0.597), while age shows a statistical significance (p = 0.000).

TABLE 7
Statistical significance testing for age and gender in the Omicron dataset.The chi-square test is used to calculate the statistical significance of age and gender.Box plot of AUC values produced by different pre-training methods using ResNet50 based on the Omicron dataset.We use 500 bootstrap samples with 300 cases to calculate the AUC which can evaluate the robustness of the model.
FIGURE 8 ROC curves of the four types of pre-training methods using ResNet50 on the Omicron dataset.Our CoTP achieves the highest 98.90% AUC.FIGURE 9

TABLE 8
The performance of MoCo-TP pre-training on the SARS-CoV-2 CT-scan dataset.

79 (99.78-99.80) *
The highest scores are shown in boldface.*Quantitative data were presented as values (95% confidence interval).FIGURE 10 Comparison of the performance of four types of pre-training methods in identifying COVID-19 pneumonia from CXR images.10.3389/fmed.2024.1360143Frontiers in Medicine 12 frontiersin.orgTABLE 9 Impact the random Poisson noise perturbation on model performance based on the Omicron dataset.